Large Number Of Unlabeled Samples Research Articles

Scalable and Inductive Semi-supervised Classifier with Sample Weighting Based on Graph Topology

Recently, graph-based semi-supervised learning (GSSL) has garnered significant interest in the realms of machine learning and pattern recognition. Although some of the proposed methods have made some progress, there are still some shortcomings that need to be overcome. There are three main limitations. First, the graphs used in these approaches are usually predefined regardless of the task at hand. Second, due to the use of graphs, almost all approaches are unable to process and consider data with a very large number of unlabeled samples. Thirdly, the imbalance of the topology of the samples is very often not taken into account. In particular, processing large datasets with GSSL might pose challenges in terms of computational resource feasibility. In this article, we present a scalable and inductive GSSL method. We broaden the scope of the graph topology imbalance paradigm to extensive databases. Second, we employ the calculated weights of the labeled sample for the label-matching term in the global objective function. This leads to a unified, scalable, semi-supervised learning model that allows simultaneous labeling of unlabeled data, projection of the feature space onto the labeling space, along with the graph matrix of anchors. In the proposed scheme, the integration of labels and features from anchors is applied for the adaptive construction of the anchor graph. Experimental results were performed on four large databases: NORB, RCV1, Covtype, and MNIST. These experiments demonstrate that the proposed method exhibits superior performance when compared to existing scalable semi-supervised learning models.

Gearbox fault diagnosis method based on improved semi-supervised MTDL and GAF

Aiming at the problem that it takes a long time and high cost to obtain complete labeled data under intelligent fault diagnosis and unlabeled data is not used. This paper proposes an improved semi-supervised mean teacher deep learning (MTDL) and Gramian angle field (GAF) fusion diagnostic method. This method fully utilizes a small number of labeled samples and a large number of unlabeled samples to deeply mine invisible fault features and potential physical correlations. At the same time, it solves the problem of losing the inter-data correlation structure when one-dimensional time series signals are used as inputs for neural networks. The GAF-MTDL method uses consistency regularization and modifies the network structure in the mean teacher algorithm into a semi-supervised deep learning model enhanced by WideResNet. The experimental results show that the proposed GAF-MTDL method saves a lot of manual labeling costs, improves the recognition accuracy and generalization ability, and can achieve excellent prediction accuracy with very little labeled data. In the end, the accuracy of planetary gear fault identification reached 98.22% under the labeling rate of 20%, and the accuracy of fault identification reached 99.98% through the verification of the bearing data set of Case Western Reserve University. The value of this research is to bring an efficient and low-cost technology to the field of industrial intelligent fault diagnosis, which can significantly improve the accuracy of fault identification.

Network_Protocol_Entity_Extraction_Based_on_Few_shot_Learning

Knowledge Graph is a new way of knowledge collections, and building a protocol knowledge graph on RFCs can help us study and analyse network protocol better. Protocol entity extraction is one of the keys to constructing the network protocol knowledge graph.Because RFC (Request For Comments) contains detailed descriptions of basic Internet communication protocols, the protocol entities of the network can be obtained from it. However, the document format and wording are not uniform, which leads to the inability to complete the extraction task of network protocol entities based on traditional rule information extraction methods. Therefore, this paper proposes a network protocol entity extraction method based on Few-Shot Learning. This method can use a very small amount of labeled samples to extract network protocol entities from a large number of unlabeled samples and maintain high recognition accuracy. This method firstly mines as many potential network protocol entities as possible in the RFC document, and secondly performs accurate re-identification of the identified potential network protocol entities. Experiments show that using 5 manually annotated RFC documents to train our model, the accuracy of network protocol entity extraction reaches 88.4%. Compared with the existing methods, this method has higher accuracy and better robustness in terms of network protocol entity extraction, and it also has better identification ability for network protocol entities that have not appeared in the training set.

Overcoming graph topology imbalance for inductive and scalable semi-supervised learning

Graph-based semi-supervised learning (GSSL) has received much attention recently. Despite some progress made in this area by some recent methods, some limitations still need to be addressed. Namely, there are two main shortcomings. First, the graphs used are very often built in advance and independently of the task at hand, using a heuristic, and generally do not represent the true topology of the data. The second shortcoming is the ability of the model to handle a very large number of unlabeled samples. This can make the GSSL solution impractical from a computational resource perspective. In this paper, we propose the Weighted Simultaneous Graph Construction and Reduced Flexible Manifold Embedding (W-SGRFME) method, which is a scalable and inductive GSSL framework. The main contributions are as follows. First, we extend the concept of graph topology imbalance to large datasets. Second, we integrate the computed weights of the labeled samples into the unified semi-supervised model. The latter jointly estimates the labels of the unlabeled samples, the mapping of the feature space to the label space, and the graph matrix of the anchor graph. Moreover, the fusion of labels and features of anchors is used to adaptively construct the graph. Experimental results on three large datasets from semi-supervised learning show the effectiveness of the proposed scalable method. These datasets are NORB, RCV1, and Covtype. Experimental results on large datasets show the superiority of the proposed method over existing scalable models.

Human Activity Recognition based on Local Linear Embedding and Geodesic Flow Kernel on Grassmann manifolds

Human Activity Recognition (HAR) plays a crucial role in various applications(e.g., medical treatment, video surveillance and sports monitoring). Transfer learning is a promising solution to cross-domain identification problems in HAR. However, existing methods usually ignore the negative transfer caused by using the features of each source domain in equal proportions, as well as the distribution difference between the source and target domains. In this paper, an HAR method based on manifold learning is proposed. Firstly, the similarity between the domain and multiple source domains is calculated using the Multi-Kernel-Maximum Mean Difference (MK-MMD), and the source domain most similar to the target domain is selected as the optimal source domain in the transfer task. Secondly, Locally Linear Embedding (LLE) is leveraged to reduce the dimensionality of both optimal source domain and target domain data to remove redundant information, and the Geodesic Flow Kernel (GFK) is utilized to project low-dimensional data into the Grassmann manifold space and reduce the distribution difference between the two domains. Finally, the source domain action training model is applied to the target domain. Three public datasets (i.e., PAMAP2, OPPORTUNITY and UCI DSADS) are utilized to validate the effectiveness of the proposed approach. Experimental results are presented to demonstrate that the proposed HAR method can predict a large number of unlabeled samples in the target domain while preserving the original data structure.

Semi-Supervised Specific Emitter Identification via Dual Consistency Regularization

Deep Learning (DL)-based Specific Emitter Identification (SEI) is a potential physical layer authentication technique for Industrial Internet-of-Things (IIoT) Security, which detects the individual emitter according to its unique signal features resulting from transmitter hardware impairments. The success of DL-based SEI often depends on sufficient training samples and the integrity of samples’ labels. The extensive deployment of wireless devices generates a huge amount of signals, but signals labeling is quite difficult and expensive with the high demand for expertise. In this paper, we present a SEI method based on Dual Consistency Regularization (DCR), which enables feature extraction and identification using a few labeled samples and a large number of unlabeled samples. With the help of pseudo-labeling, we leverage consistency between the predicted class distribution of weakly-augmented unlabeled training samples and that of strongly-augmented training unlabeled samples, and consistency between semantic feature distribution of labeled samples and that of pseudo-labeled samples, which takes the unlabeled samples into account to model parameter tuning for a more accurate emitter identification. Extensive numerical results demonstrate that compared with well-known semi-supervised learning-based SEI methods, our method obtains 99.77% identification accuracy on a Wi-Fi dataset and 90.10% identification accuracy on an Automatic Dependent Surveillance-Broadcast (ADS-B) dataset when only 10% of training samples are labeled, and improves the identification accuracy on the Wi-Fi dataset and the ADS-B dataset by more than 19.07% and 5.30%, respectively. Our codes are available at https://github.com/lovelymimola/DCR-Based-SemiSEI.

A novel semi-supervised fault diagnosis method for chillers based on neighbor-optimized graph convolutional network

The fault diagnosis of chillers is of great significance in reducing building energy consumption and extending the operational lifespan of refrigeration equipment. Several popular machine learning-based fault diagnosis methods rely heavily on many labeled samples. However, such samples are difficult to obtain in practice due to sparse fault data and high labeling costs. This limits the application of ML-based fault diagnosis methods based on supervised learning. To reduce the dependence on labeled samples, this paper proposes a novel chiller fault diagnosis method based on neighbor-optimized graph convolutional network. The method improves the utilization efficiency of unlabeled samples by mining the spatio-temporal relationship between a large number of unlabeled samples and a limited number of labeled samples. And it dynamically adjusts the graph's structure by optimizing the number of adjacent samples in the correlation graph to obtain better diagnostic results. Its effectiveness is validated on the authoritative dataset ASHRAE RP-1043 and a more challenging dataset of real-world chillers in a building. Experimental results show that the proposed method can achieve better diagnostic performance than the state-of-the-art methods.

Semi-supervised fault diagnosis of gearbox based on feature pre-extraction mechanism and improved generative adversarial networks under limited labeled samples and noise environment

Gearboxes are the most widely used component to transfer speed and power in many industries, and high precision gearbox fault diagnosis (FD) is pretty crucial for ensuring the safe operation of the machine. However, traditional FD methods often need a great quantity of labeled data, and are prone to noise interference in practical work, resulting in a relatively low diagnosis accuracy. With the intention of overcoming these problems, this paper proposes a semi-supervised FD approach based on feature pre-extraction mechanism and improved generative adversarial network (IGAN). First, the data is preprocessed by the feature pre-extraction mechanism based on wavelet transform. Then, limited labeled samples and a large number of unlabeled samples are sent to the IGAN model. Finally, two typical gearbox fault datasets are utilized to evaluate the feasibility and effectiveness of the proposed approach in limited labeled samples and noise environment. Trial results denote that the proposed approach has better diagnosis accuracy and anti-noise robustness than other approaches.

A self-training algorithm based on the two-stage data editing method with mass-based dissimilarity

A self-training algorithm is a classical semi-supervised learning algorithm that uses a small number of labeled samples and a large number of unlabeled samples to train a classifier. However, the existing self-training algorithms consider only the geometric distance between data while ignoring the data distribution when calculating the similarity between samples. In addition, misclassified samples can severely affect the performance of a self-training algorithm. To address the above two problems, this paper proposes a self-training algorithm based on data editing with mass-based dissimilarity (STDEMB). First, the mass matrix with the mass-based dissimilarity is obtained, and then the mass-based local density of each sample is determined based on its k nearest neighbors. Inspired by density peak clustering (DPC), this study designs a prototype tree based on the prototype concept. In addition, an efficient two-stage data editing algorithm is developed to edit misclassified samples and efficiently select high-confidence samples during the self-training process. The proposed STDEMB algorithm is verified by experiments using accuracy and F-score as evaluation metrics. The experimental results on 18 benchmark datasets demonstrate the effectiveness of the proposed STDEMB algorithm.

A Transformer-Based Contrastive Semi-Supervised Learning Framework for Automatic Modulation Recognition

The application of deep learning improves the processing speed and the accuracy of automatic modulation recognition (AMR). As a result, it realizes intelligent spectrum management and electronic reconnaissance. However, deep learning-aided AMR usually requires a large number of labeled samples to obtain a reliable neural network model. In practical applications, due to economic costs and privacy constraints, there is a small number of labeled samples but a large number of unlabeled samples. This paper proposes a Transformer-based contrastive semi-supervised learning framework for AMR. First, self-supervised contrastive pre-training of the Transformer-based encoder is completed using unlabeled samples, and data augmentation is realized through time warping. Then, the pre-trained encoder and a randomly initialized classifier are fine-tuned using labeled samples, and hierarchical learning rates are employed to ensure classification accuracy. Considering the problems of applying Transformer to AMR, a convolutional transformer deep neural network is proposed, which involves convolutional embedding, attention bias, and attention pooling. In experiments, the feasibility of the framework is analyzed through linear evaluation of the framework components on the RML2016.10a dataset. Also, the proposed framework is compared with existing semi-supervised methods on RML2016.10a and RML2016.10b datasets to verify its superiority and stability.

Super-resolution image visual quality assessment based on structure–texture features

Assessing the visual quality of super-resolution images (SRIs) is crucial for advancing algorithm development, but it remains an unsolved problem. In this paper, we present a novel reduced-reference image quality assessment (RR-IQA) method specifically suited for evaluating SRIs. Our approach leverages information from the input low-resolution (LR) image as a reference signal to extract features that are most relevant to modeling the visual quality of SRIs. We analyze the artifact characteristics of SRIs and demonstrate that features describing edge orientations, high frequency components, and textures are the most important for this task. To extract these features, we first perform structure–texture decompositions (STD) on both the SRI and its LR input, then obtain the edge orientation feature through a traditional hand-crafted approach, and deep neural networks to extract features related to high frequency components and textures. We employ a shallow multilayer perceptron (MLP) to predict an image quality score based on these quality-relevant features. To improve feature representation ability and prevent overfitting, we pretrain the feature extraction module using a large number of unlabeled samples, which accounts for over 99.8% of the total model parameters. We use precious samples with mean opinion score (MOS) labels to train a high-quality shallow MLP predictor. Our experimental results show that the proposed method outperforms classical and state-of-the-art models.

SEML: Self-Supervised Information-Enhanced Meta-learning for Few-Shot Text Classification

Training a deep-learning text classification model usually requires a large amount of labeled data, yet labeling data are usually labor-intensive and time-consuming. Few-shot text classification focuses on predicting unknown samples using only a few labeled samples. Recently, metric-based meta-learning methods have achieved promising results in few-shot text classification. They use episodic training in labeled samples to enhance the model’s generalization ability. However, existing models only focus on learning from a few labeled samples but neglect to learn from a large number of unlabeled samples. In this paper, we exploit the knowledge learned by the model in unlabeled samples to improve the generalization performance of the meta-network. Specifically, we introduce a novel knowledge distillation method that expands and enriches the meta-learning representation with self-supervised information. Meanwhile, we design a graph aggregation method that efficiently interacts the query set information with the support set information in each task and outputs a more discriminative representation. We conducted experiments on three public few-shot text classification datasets. The experimental results show that our model performs better than the state-of-the-art models in 5-way 1-shot and 5-way 5-shot cases.

Semi-supervised interlayer intelligent recognition method

The development and distribution of interlayers in sandstones can lead to increased heterogeneity in the formation within the reservoir, which further affects the movement of fluids in the formation. Therefore, it is essential to accurately evaluate the interlayers in sandstones, as the precise evaluation of interlayers in sandstones is of great importance for identifying the distribution of underground fluid systems. The logging data of interlayers are inadequate for traditional Machine Learning training due to their low measurement proportion compared to the conventional layers. In logging data, the amount of data in interlayers is significantly smaller than that of conventional reservoirs. Traditional machine learning models are mostly based on samples with balanced distribution. By contrast, semi-supervised learning requires a small number of labeled samples for learning, and then combines a large number of unlabeled samples for modeling. To verify the feasibility of semi-supervised learning in the identification of interlayers, the Donghe sandstone section of the H oilfield was taken as an example. First, the core analysis results were used to label the logging data; then, to uncover more response information that can characterize the interlayers on the logging curve, multiple features were extracted to construct cross features. Finally, an improved model based on autoencoders—probabilistic autoencoder (PAE)—is proposed to address the issue of interlayer recognition for imbalanced samples. The PAE model can calculate the probability of belonging to a different class for unlabeled samples, and classify new samples according to the maximum probability. Experimental results show that, compared with traditional machine learning methods and ensemble learning methods, PAE achieves higher recognition accuracy and better generalization performance by updating the algorithm, and can be used as a simple and fast method for interlayer recognition. The results of the algorithm demonstrate that the semi-supervised method is of great significance for exploring and developing complex heterogeneous oil reservoirs. The research results are of great importance for exploring and developing complex heterogeneous oil reservoirs.

Semi-Supervised Deep Kernel Active Learning for Material Removal Rate Prediction in Chemical Mechanical Planarization.

The material removal rate (MRR) is an important variable but difficult to measure in the chemical-mechanical planarization (CMP) process. Most data-based virtual metrology (VM) methods ignore the large number of unlabeled samples, resulting in a waste of information. In this paper, the semi-supervised deep kernel active learning (SSDKAL) model is proposed. Clustering-based phase partition and phase-matching algorithms are used for the initial feature extraction, and a deep network is used to replace the kernel of Gaussian process regression so as to extract hidden deep features. Semi-supervised regression and active learning sample selection strategies are applied to make full use of information on the unlabeled samples. The experimental results of the CMP process dataset validate the effectiveness of the proposed method. Compared with supervised regression and co-training-based semi-supervised regression algorithms, the proposed model has a lower mean square error with different labeled sample proportions. Compared with other frameworks proposed in the literature, such as physics-based VM models, Gaussian-process-based regression models, and stacking models, the proposed method achieves better prediction results without using all the labeled samples.

Semi-Supervised Remote Sensing Image Semantic Segmentation Method Based on Deep Learning

In this paper, we study the semi-supervised semantic segmentation problem via limited labeled samples and a large number of unlabeled samples. We propose a self-learning semi-supervised approach for the semantic segmentation of high-resolution remote sensing images. Our approach uses two networks (UNet and DeepLabV3) to predict the labels of the same unlabeled sample, and the pseudo labels samples with high prediction consistency are added to the training samples to improve the accuracy of semantic segmentation under the condition of limited labeled samples. Our method expands training data samples by using unlabeled data samples with pseudo labels. In order to verify the effectiveness of the proposed method, some experiments were conducted on the improved ISPRS Vaihingen 2D Semantic Labeling dataset using the method that we proposed. We focus on the extraction of forest and vegetation information and focus on the impact of a large number of unlabeled samples on the precision of semantic segmentation, we combine water, surface, buildings, cars, and background into one category and named others, and we call the changed dataset the improved ISPRS Vaihingen dataset. The experimental results show that the proposed method can effectively improve the semantic segmentation accuracy of high-scoring remote sensing images with limited samples than common deep semi-supervised learning.

Hyperspectral Image Classification Based on Unsupervised Regularization

Due to the powerful feature expression ability of deep learning and its end-to-end non-linear mapping relationship, deep learning-based methods have become the mainstream method for hyperspectral image classification tasks. However, the accuracy of deep learning methods greatly depends on the use of a large number of labeled samples to train the model. Also, hyperspectral images have few labeled samples and unbalanced categories, which makes the depth model prone to overfitting, which seriously affects the classification accuracy. Therefore, how to alleviate the overfitting phenomenon caused by small samples in the classification problem based on deep learning is still a problem that needs to be solved. Considering that it is relatively easier to obtain a large number of unlabeled samples in the field of remote sensing, making full use of the unsupervised information learned from unlabeled data can regularize the supervised classification model, which can effectively alleviate the overfitting phenomenon caused by the small samples problem. In the supervised training process, unsupervised information from the overall distribution of the sample is introduced to guide the regularization of the model, so as to realize the effective classification of the data in the case of a small number of labeled samples. Experimental results demonstrate the effectiveness of the proposed method in terms of hyperspectral image (HSI) classification with few training samples.

Uncertainty-Guided Semi-Supervised Few-Shot Class-Incremental Learning With Knowledge Distillation

Class-Incremental Learning (CIL) aims at incrementally learning novel classes without forgetting old ones. This capability becomes more challenging when novel tasks contain one or a few labeled training samples, which leads to a more practical learning scenario, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., Few-Shot Class-Incremental Learning (FSCIL). The dilemma on FSCIL lies in serious overfitting and exacerbated catastrophic forgetting caused by the limited training data from novel classes. In this paper, excited by the easy accessibility of unlabeled data, we conduct a pioneering work and focus on a Semi-Supervised Few-Shot Class-Incremental Learning (Semi-FSCIL) problem, which requires the model incrementally to learn new classes from extremely limited labeled samples and a large number of unlabeled samples. To address this problem, a simple but efficient framework is first constructed based on the knowledge distillation technique to alleviate catastrophic forgetting. To efficiently mitigate the overfitting problem on novel categories with unlabeled data, uncertainty-guided semi-supervised learning is incorporated into this framework to select unlabeled samples into incremental learning sessions considering the model uncertainty. This process provides extra reliable supervision for the distillation process and contributes to better formulating the class means. Our extensive experiments on CIFAR100, miniImageNet and CUB200 datasets demonstrate the promising performance of our proposed method, and define baselines in this new research direction.

Research on the Guidance of Youth Labor Education Based on the "Combination of Education and Production Labor" Program Based on the Deep Learning Model.

At present, there is a lack of research on Marx's idea of “combining education and productive labor” and its guiding significance for youth labor education, and no effective teaching model has been formed. In response to this problem, this study proposes a semi-supervised deep learning model based on u-wordMixup (SD-uwM). When there is a shortage of labeled samples, semi-supervised learning uses a large number of unlabeled samples to solve the problem of labeling bottlenecks. However, since the unlabeled samples and labeled samples come from different fields, there may be quality problems in the unlabeled samples, which makes the generalization ability of the model worse., resulting in a decrease in classification accuracy. The model uses the u-wordMixup method to perform data augmentation on unlabeled samples. Under the constraints of supervised cross-entropy and unsupervised consistency loss, it can improve the quality of unlabeled samples and reduce overfitting. The comparative experimental results on the AGNews, THUCNews, and 20Newsgroups data sets show that the proposed method can improve the generalization ability of the model and also effectively improve the time performance. The study found that the SD-uwM model uses the u-wordMixup method to enhance the unlabeled samples and combines the idea of the Mean Teacher model, which can significantly improve the text classification performance. The SD-uwM model can improve the generalization ability and time performance of the model, respectively, 86.4 ± 1.3 and 90.5 ± 1.3. Therefore, the use of SD-uwM in Marx's program is of great practical significance for the guidance process of youth labor education.

Positive-Unlabeled Learning for Network Link Prediction

Link prediction is an important problem in network data mining, which is dedicated to predicting the potential relationship between nodes in the network. Normally, network link prediction based on supervised classification will be trained on a dataset consisting of a set of positive samples and a set of negative samples. However, well-labeled training datasets with positive and negative annotations are always inadequate in real-world scenarios, and the datasets contain a large number of unlabeled samples that may hinder the performance of the model. To address this problem, we propose a positive-unlabeled learning framework with network representation for network link prediction only using positive samples and unlabeled samples. We first learn representation vectors of nodes using a network representation method. Next, we concatenate representation vectors of node pairs and then feed them into different classifiers to predict whether the link exists or not. To alleviate data imbalance and enhance the prediction precision, we adopt three types of positive-unlabeled (PU) learning strategies to improve the prediction performance using traditional classifier estimation, bagging strategy and reliable negative sampling. We conduct experiments on three datasets to compare different PU learning methods and discuss their influence on the prediction results. The experimental results demonstrate that PU learning has a positive impact on predictive performances and the promotion effects vary with different network structures.

An Active Learning Algorithm Based on the Distribution Principle of Bhattacharyya Distance

Active learning is a method that can actively select examples with much information from a large number of unlabeled samples to query labeled by experts, so as to obtain a high-precision classifier with a small number of samples. Most of the current research uses the basic principles to optimize the classifier at each iteration, but the batch query with the largest amount of information in each round does not represent the overall distribution of the sample, that is, it may fall into partial optimization and ignore the whole, which will may affect or reduce its accuracy. In order to solve this problem, a special distance measurement method—Bhattacharyya Distance—is used in this paper. By using this distance and designing a new set of query decision logic, we can improve the accuracy of the model. Our method embodies the query of the samples with the most representative distribution and the largest amount of information to realize the classification task based on a small number of samples. We perform theoretical proofs and experimental analysis. Finally, we use different data sets and compare them with other classification algorithms to evaluate the performance and efficiency of our algorithm.